REMARKS 

Claims 1, 4-22, and 24-36 are pending in the application, with claims 1, 22, 25, 26, 27, 
and 32 being the independent claims. Based on the following remarks, Applicant respectfully 
requests that the Examiner reconsider all outstanding rejections and that they be withdrawn. 

Interview 

1 . Applicant thanks Examiner Tung Vo for the interview with Dr. Alan Lipton and 
Applicant's representative on September 13, 2006 to discuss claim 1, the Merheim reference, and 
the Araki reference. 

Anticipation Rejection Based on Merheim 

3. On pages 2-6 in section 3, the Office Action rejected claims 1,4-11, 13-22, and 
24-36 under 35 U.S.C. § 102(e) as being anticipated by U.S. Patent Application Publication No. 
2002/0135483 to Merheim et al. (hereinafter Merheim) . Applicant respectfully traverses the 
rejection. 

Claim 1, as amended, recites a computer-readable medium comprising software for a 
video surveillance system, comprising code segments for operating the video surveillance system 
based on video primitives, wherein the code segments for operating the video surveillance system 
comprise: code segments for identifying one or more user-defined event discriminators ; code 
segments for extracting video primitives from a video; and code segments for extracting event 
occurrences from the video primitives using at least one of the one or more user-defined 
event discriminators. 
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Support for the amendment to the claim of " user-defined event discriminators" may be 
found in the specification at, for example, paragraphs [66], [71], [74], [79], [80], [94], and [153] 
and Figure 12. Support for the amendment to the claim of "video primitives from a video" may 
be found in the specification at, for example, paragraphs [43], [65], [71]-[73], [77], [104]-[106], 
and [148]. 

Merheim fails to teach claim 1 for several reasons. First, Merheim fails to teach 
identifying one or more event discriminators . As discussed in the specification, event 
discriminators provide an operator with maximum flexibility in configuring a video surveillance 
system. Specification, paragraph [66], As the Office Action may not have realized, the 
specification provides a definition for "event discriminator." As defined in the specification, an 
event discriminator refers to one or more objects optionally interacting with one or more 
spatial attributes and/or one or more temporal attributes . Specification, paragraph [80]. As 
defined in the specification, an "object" refers to an item of interest in a video. Examples of an 
object include a person, a vehicle, an animal, and a physical subject. Specification, [45]. Event 
discriminators may be identified with one or more objects, along with one or more optional 
spatial attributes, and/or one or more optional temporal attributes. For example, an operator may 
identify an event discriminator for "loitering" as a "person" object in the "automatic teller 
machine" space for "longer than 15 minutes" and "between 10:00 p.m. and 6:00 a.m." 
Specification, paragraph [66]. 

In rejecting claim 1, the Office Action aligns the recited identifying one or more event 
discriminators with a low-level image processing algorithm for performing object segmentation. 
Merheim, Figure 3, blocks 100, 1 10, and 120; paragraphs [0013], [0016], and [0061]. In block 
100 of Figure 3 of Merheim , an image is obtained. In block 1 10, the image is digitized and a 
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divergence image is obtained by computing a difference image between the digitized image and a 
reference image (or background frame). In block 120, areas are extracted that have changed 
between the digitized image and the reference image. Merheim , paragraph [0061]. 

This low-level image processing algorithm taught by Merheim and relied upon by the 
Office Action may be aligned with a background subtraction process that might be used to 
extract objects in block 52 in Figure 5 of the specification. In block 52 of the specification, 
objects are detected via change, and any change detection algorithm for detecting changes from a 
background model can be used for this block. Specification, paragraph [108]. The low-level 
image processing algorithm taught by Merheim and relied upon by the Office Action may be 
aligned with detecting objects via change in block 52 of the specification and not with 
identifying one or more event discriminators. Such a background subtraction process as 
disclosed by Merheim is unrelated to identifying one or more event discriminators. As such, 
Merheim fails to teach identifying one or more event discriminators. 

Second, Merheim fails to teach extracting video primitives . As the Office Action may 
not have realized, the specification provides a definition for "video primitive." As defined in the 
specification, a "video primitive" refers to an observable attribute of an object viewed in a 
video feed. Specification, [80]. Examples of video primitives provided in the specification 
include: an identification of an object belonging to a particular category or class; a dimensional 
attribute of an object; a chromatic attribute of an object; a pattern attribute of an object; a 
measure of the rigidity of an object; a motion of an object that can be automatically detected and 
tracked for some period of time; and a property of salient motion of an object. Specification, 
[80]-[91]. These examples of video primitives are basically things that a human can observe 
about an object. Further, the specification provides a definition for "object," where "object" is 
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used in the definition of "video primitive." As defined in the specification, an "object" refers to 
an item of interest in a video. Specification, [45]. Examples of an object include a person, a 
vehicle, an animal, and a physical subject. Specification, [45]. 

In rejecting claim 1, the Office Action aligns the recited extracting video primitives with 
a low-level coding scheme in Merheim . Merheim . Figure 3, blocks 130, 140, and 150; paragraph 
[0061]. In block 130 in Figure 3 of Merheim. the digitized image is filtered to remove noise. In 
block 140, an "object" is extracted. In block 150, the "object" of the image is stored using 
polygonization. Merheim , paragraphs [0061]- [0063]. Perhaps, the Office Action is confused by 
the use of the word "object" in Merheim and the use of the word "object" in the specification. 
Although the same word is used, two different concepts are being referenced. In Merheim, the 
word "object" refers to an unidentifiable portion of an image (i.e.. a frame) having certain 
characteristics (e.g., coordinates, size, outline, average intensity, circumference, and intensity 
variations). Merheim. paragraph [0061]. In contrast, as discussed above, in the specification, the 
word "object" refers to an item of interest in a video, where examples of an object include a 
person, a vehicle, an animal, and a physical subject. Specification, [45]. The "object" as used in 
Merheim is more in line with a "blob" as used in the specification. As discussed in the 
specification, a blob refers to any object in a frame . Specification, [111]. 

The low-level coding scheme taught by Merheim and relied upon by the Office Action 
may be aligned with a blob detection process that might be used to in block 53 in Figure 5 of the 
specification. In block 53 of the specification, blobs are generated, and any technique for 
generating blobs can be used for this block. Specification, paragraph [111]. The low-level 
image processing algorithm taught by Merheim and relied upon by the Office Action may be 
aligned with generating blobs in block 53 of the specification and not with extracting video 
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primitives. Such a coding scheme as disclosed by Merheim is unrelated to extracting video 
primitives. As such, Merheim fails to teach extracting video primitives. 

Third, Merheim fails to teach extracting event occurrences from the video primitives 
using at least one of the one or more event discriminators. As discussed in the specification, the 
event discriminators are used to filter the video primitives to determine if any event occurrences 
occurred. Specification, paragraph [118]. As an example discussed in the specification, an event 
discriminator can be looking for a "wrong way" event as defined by a person traveling the 
"wrong way" into an area between 9:00 a.m. and 5:00 p.m. The exemplary event discriminator 
then checks all video primitives being generated and determines if any video primitives exist 
which have the following properties: a timestamp between 9:00 a.m. and 5:00 p.m., a 
classification of "person" or "group of people", a position inside the area, and a "wrong" 
direction of motion. If a video primitive matches the properties of the event discriminator, an 
event occurrence occurred. Specification, paragraph [118]. 

In rejecting claim 1, the Office Action aligns the recited extracting event occurrences 
from the video primitives using at least one of the one or more event discriminators with a low- 
level object tracking algorithm that uses object matching of Merheim . Merheim , Figure 3, block 
160; Figure 7, blocks 200, 210, and 220; paragraphs [0064]-[0065]. In block 200 of Figure 7 of 
Merheim, an object in the current image (i.e., a "blob" as taught in the specification) is compared 
to each of the objects in the previous image (i.e., "blobs"). In block 210, a simple match score is 
calculated for each comparison. In block 220, a threshold is applied to each of the match scores 
to determine if the object in the current image matches any of the objects in the previous image. 
Merheim, paragraph [0065]. 
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The low-level object matching and tracking taught by Merheim and relied upon by the 
Office Action may be aligned with a blob tracking process that might be used to in block 54 in 
Figure 5 of the specification. In block 54 of the specification, blobs are tracked, and any 
technique for tracking blobs can be used for this block. Specification, paragraph [112]. The low- 
level image processing algorithm taught by Merheim and relied upon by the Office Action may 
be aligned with tracking blobs in block 54 of the specification and not with extracting event 
occurrences from the video primitives using at least one of the one or more event discriminators. 
Such a tracking scheme as disclosed by Merheim is unrelated to extracting event occurrences 
from the video primitives using at least one of the one or more event discriminators. As such, 
Merheim fails to teach extracting event occurrences from the video primitives using at least one 
of the one or more event discriminators. 

Therefore, based on the above arguments, Merheim fails to teach claim 1 . 

Claims 4-11,13-21, and 29-31 depend variously from claim 1, and are allowable as being 
dependent from an allowable claim. 

Claim 22 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claim 24 depends from claim 22, and is allowable as being dependent from an allowable 

claim. 

Claim 25 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claim 26 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 
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Claim 27 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claim 28 depends from claim 27, and is allowable as being dependent from an allowable 

claim. 

Claim 32 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claims 33-36 depend from claim 32, and are allowable as being dependent from an 
allowable claim. 

Obviousness Rejection Based on Araki and Ito 

3. On pages 6-10 in section 5, the Office Action rejected claims 1, 4-22, and 24-36 
under 35 U.S.C. § 102(e) as being anticipated by U.S. Patent No. 4,737,847 to Araki et al. 
(hereinafter Araki ) in view of U.S. Patent No. 6,404,455 to Ito et al. (hereinafter Ito). Applicant 
respectfully traverses the rejection. 

The combination of Araki and Ito fails to teach claim 1 for several reasons. First, the 
combination fails to teach identifying one or more user-defined event discriminators . As 
discussed above, event discriminators provide an operator with maximum flexibility in 
configuring a video surveillance system. Specification, paragraph [66]. As the Office Action 
may not have realized, the specification provides a definition for "event discriminator." As 
defined in the specification, an event discriminator refers to one or more objects optionally 
interacting with one or more spatial attributes and/or one or more temporal attributes . 
Specification, paragraph [80], As defined in the specification, an "object" refers to an item of 
interest in a video. Examples of an object include a person, a vehicle, an animal, and a physical 
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subject. Specification, [45]. Event discriminators may be identified with one or more objects, 
along with one or more optional spatial attributes, and/or one or more optional temporal 
attributes. Moreover, the event discriminators are user defined . See Specification, e.g., 
paragraphs [66], [71], [74], [79], [80], [94], and [153] and Figure 12. For example, an operator 
may identify an event discriminator for "loitering" as a "person" object in the "automatic teller 
machine" space for "longer than 15 minutes" and "between 10:00 p.m. and 6:00 a.m." 
Specification, paragraph [66]. 

In rejecting claim 1, the Office Action aligns the recited event discriminators with the 
setting area of Araki . Araki, Figure 88 ("Setting Area"); Figure 46, items 296, 297, 299, and 
300; column 19, line 55 to column 20, line 8. To the best understanding of the Applicant, Araki 
is a difficult patent to parse. The discussion of Figure 88 in Araki is very sparse. Araki, column 
29, lines 5-10. Items 296 and 297 in Figure 46 relate back to items 26 and 27 in Figure 4. Araki, 
column 20, lines 35-42. In Figure 4, a detection area setting means 26 is used to divide a 
monitoring zone into several areas of different warning levels according to a demanded warning 
degree. Araki, column 7, lines 4-12. The divided areas of the monitoring zone set by the 
detection area setting means 26 are stored in a detection area means 27, which further provides 
them to an abnormality discrimination means 22. Araki , column 7, lines 12-21. 

The divided areas of the monitoring zone set by the detection area setting means 26 of 
Araki are not the same as the recited user-defined event discriminators. As defined in the 
specification, an event discriminator refers to one or more objects optionally interacting with one 
or more spatial attributes and/or one or more temporal attributes. Specification, paragraph [80]. 
The divided areas of the monitoring zone set by the detection area setting means 26 do not refer 
to one or more objects, as defined by a user, optionally interacting with one or more spatial 
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attributes and/or one or more temporal attributes. Moreover, Ito fails to overcome the failings of 
Araki . Thus, Araki and Ito fail to teach identifying one or more user-defined event 
discriminators. 

Second, Araki and Ito fail to teach extracting video primitives . As discussed above, the 
specification provides a definition for "video primitive." As defined in the specification, a 
"video primitive" refers to an observable attribute of an object viewed in a video feed. 
Specification, [80]. Examples of video primitives provided in the specification include: an 
identification of an object belonging to a particular category or class; a dimensional attribute of 
an object; a chromatic attribute of an object; a pattern attribute of an object; a measure of the 
rigidity of an object; a motion of an object that can be automatically detected and tracked for 
some period of time; and a property of salient motion of an object. Specification, [80]-[91], 
These examples of video primitives are basically things that a human can observe about an 
object. Further, the specification provides a definition for "object," where "object" is used in the 
definition of "video primitive." As defined in the specification, an "object" refers to an item of 
interest in a video. Specification, [45]. Examples of an object include a person, a vehicle, an 
animal, and a physical subject. Specification, [45]. 

In rejecting claim 1, the Office Action aligns the recited event discriminators with 
extracting video primitives with a low-level coding scheme in Araki . Araki , Figure 88 ("Picture 
Processing"); Figure 46, item 291; Figure 2 (object extraction and tracking). The picture 
processing 291 in Figure 46 is the same as the picture processing 21 in Figure 4, which is the 
same as the picture processing 1 1 in Figure 1. Araki , column 20, lines 35-42; column 6, lines 61- 
65. The picture processing 1 1 is further depicted in Figure 2 and includes subtraction of an input 



- 17- 



Application No. 09/987,707 



picture from a reference picture, object extraction and frame tracking. Araki, column 5, lines 28- 
59. 

The low-level coding scheme taught by Araki and relied upon by the Office Action may 
be aligned with the low-level processing of the specification. The subtraction of an input picture 
from a reference picture of Araki may be aligned with a background subtraction process that 
might be used to extract objects in block 52 in Figure 5 of the specification. The object 
extraction of Araki may be aligned with a blob detection process that might be used in block 53 
in Figure 5 of the specification. The frame tracking of Araki may be aligned with a blob tracking 
process that might be used to in block 54 in Figure 5 of the specification. The low-level image 
processing algorithm taught by Araki and relied upon by the Office Action may be aligned with 
extracting objects in block 52, generating blobs in block 53, and tracking blobs in block 54 in 
Figure 5 of the specification of the specification and not with extracting video primitives. Such a 
processing scheme as disclosed by Araki is unrelated to extracting video primitives. Moreover, 
Ito fails to overcome the failings of Araki . Thus, Araki and Ito fail to teach extracting video 
primitives. 

Third, Araki fails to teach extracting event occurrences from the video primitives using 
at least one of the one or more user-defined event discriminators. As discussed above, the event 
discriminators are used to filter the video primitives to determine if any event occurrences 
occurred. Specification, paragraph [118]. As an example discussed in the specification, an event 
discriminator can be looking for a "wrong way" event as defined by a person traveling the 
"wrong way" into an area between 9:00 a.m. and 5:00 p.m. The exemplary event discriminator 
then checks all video primitives being generated and determines if any video primitives exist 
which have the following properties: a timestamp between 9:00 a.m. and 5:00 p.m., a 
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classification of "person" or "group of people", a position inside the area, and a "wrong" 
direction of motion. If a video primitive matches the properties of the event discriminator, an 
event occurrence occurred. Specification, paragraph [118]. 

In rejecting claim 1, the Office Action aligns the recited extracting event occurrences 
from the video primitives using at least one of the one or more user-defined event discriminators 
with the abnormality discrimination means 292 of Araki . Araki, Figure 88 ("Intruder 
Discrimination"); Figure 46, item 292). However, as discussed above, Araki fails to teach both 
user-defined event discriminators and video primitives. As such, Araki cannot teach extracting 
event occurrences from the video primitives using at least one of the one or more user-defined 
event discriminators. Moreover, Ito fails to overcome the failings of Araki . Thus, Araki and Ito 
fail to teach extracting event occurrences from the video primitives using at least one of the one 
or more user-defined event discriminators. 

Therefore, based on the above arguments, the combination of Araki and Ito fails to teach 
claim 1. 

Claims 4-21 and 29-31 depend variously from claim 1, and are allowable as being 
dependent from an allowable claim. 

Claim 22 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claim 24 depends from claim 22, and is allowable as being dependent from an allowable 

claim. 

Claim 25 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 
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Claim 26 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claim 27 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claim 28 depends from claim 27, and is allowable as being dependent from an allowable 

claim. 

Claim 32 recites similar features as claim 1 and is allowable for at least similar reasons as 
discussed above with respect to claim 1 . 

Claims 33-36 depend from claim 32, and are allowable as being dependent from an 
allowable claim. 

CONCLUSION 

All of the stated grounds of rejection have been properly traversed, accommodated, or 
rendered moot. Applicant therefore respectfully requests that all presently outstanding rejections 
be reconsidered and that they be withdrawn. Applicant believes that a full and complete reply 
has been made to the outstanding Office Action and, as such, the present application is in 
condition for allowance. If the Examiner believes, for any reason, that personal communication 
will expedite prosecution of this application, the Examiner is hereby invited to telephone the 
undersigned at the number provided. 
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Prompt and favorable consideration of this Amendment is respectfully requested. 



Respectfully submitted, 



Michael A. Sartori, Ph.D. 

Registration No.: 41,289 
Venable LLP 
P.O. Box 34385 
Washington, D.C. 20043-9998 
Telephone: (202) 344-4000 
Telefax: (202)344-8300 



Date: Se ptember 21. 2006 
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